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Abstract 

A noisy network coding scheme for sending multiple sources over a general noisy network is 
presented. For multi-source multicast networks, the scheme naturally extends both network coding 
over noiseless networks by Ahlswede, Cai, Li, and Yeung, and compress-forward coding for the relay 
channel by Cover and El Gamal to general discrete memoryless and Gaussian networks. The scheme also 
recovers as special cases the results on coding for wireless relay networks and deterministic networks 
by Avestimehr, Diggavi, and Tse, and coding for wireless erasure networks by Dana, Gowaikar, Palanki, 
Hassibi, and Effros. The scheme involves message repetition coding, relay signal compression, and 
simultaneous decoding. Unlike previous compress-forward schemes, where independent messages are 
sent over multiple blocks, the same message is sent multiple times using independent codebooks as in the 
network coding scheme for cyclic networks. Furthermore, the relays do not use Wyner-Ziv binning as in 
previous compress-forward schemes, and each decoder performs simultaneous joint typicality decoding 
on the received signals from all the blocks without explicitly decoding the compression indices. A 
consequence of this new scheme is that achievability is proved simply and more generally without 
resorting to time expansion to extend results for acyclic networks to networks with cycles. The noisy 
network coding scheme is then extended to general multi-source networks by combining it with decoding 
techniques for interference channels. For the Gaussian multicast network, noisy network coding improves 
the previously established gap to the cutset bound. We also demonstrate through two popular AWGN 
network examples that noisy network coding can outperform conventional compress-forward, amplify- 
forward, and hash-forward coding schemes. 
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I. Introduction 

Consider the iV-node discrete memoryless network depicted in Figure [TJ Each node wishes to send a 
message to a set of destination nodes while acting as a relay for messages from other nodes. What is 
the capacity region of this network, that is, the set of rates at which the nodes can reliably communicate 
their messages? What is the coding scheme that achieves the capacity region? These questions are at the 
heart of network information theory, yet complete answers remain elusive. 

M k -> (X k ,Y k ) 




Fig. 1. An iV-node discrete memoryless network. 

Some progress has been made toward answering these questions in the past forty years. In [H, J2l, a 
general cutset outer bound on the capacity region of this network was established. This bound generalizes 
the max-flow min-cut theorem for noiseless single-source unicast networks O, [|4l, and has been shown 
to be tight for several other classes of networks. 

In their seminal paper on network coding |5], Ahlswede, Cai, Li, and Yeung showed that the capacity 
of noiseless single-source multicast networks coincides with the cutset bound, thus generalizing the max- 
flow min-cut theorem to multiple destinations. Each relay in the network coding scheme sends a function 
of its incoming signals over each outgoing link instead of simply forwarding incoming signals. Their 
proof of the network coding theorem is done in two steps. For acyclic networks, relay mappings are 
randomly generated and they show that the message is correctly decoded with high probability provided 
the rate is below the cutset bound. This proof is then extended to cyclic networks by constructing an 
acyclic time-expanded network and relating achievable rates and codes for the time-expanded network to 
those for the original cyclic network. 

The network coding theorem has been extended in several directions. Dana, Gowaikar, Palanki, Hassibi, 
and Effros studied the multiple-source multicast erasure network as a simple model for a wireless 
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data network with packet loss. They showed that the capacity region coincides with the cutset bound 
and is achieved via network coding. Ratnakar and Kramer Q extended network coding to characterize 
the multicast capacity for single-source deterministic networks with broadcast but no interference at the 
receivers. Avestimehr, Diggavi, and Tse [8] further extended this result to deterministic networks with 
broadcast and interference to obtain a lower bound on capacity that coincides with the cutset bound when 
the channel output is a linear function of input signals over a finite field. Their proof is again done in 
two steps. As in the original proof of the network coding theorem, random coding is used to establish 
the lower bound for layered deterministic networks. A time-expansion technique is then used to extend 
the capacity lower bound to arbitrary nonlayered deterministic networks. 

In an earlier and seemingly unrelated line of investigation, van der Meulen (9| introduced the relay 
channel with a single source X\, single destination Y 3 , and single relay with transmitter-receiver pair 
(X 2 ,Y 2 ). Although the capacity for this channel is still not known in general, several nontrivial upper 
and lower bounds have been developed. In |[T0l . Cover and El Gamal proposed the compress-forward 
coding scheme in which the relay compresses its noisy observation of the source signal and forwards 
the compressed description to the destination. Despite its simplicity, compress-forward was shown to 
be optimal for classes of deterministic [11] and modulo-sum |[T2l relay channels. The Cover-El Gamal 
compress-forward lower bound on capacity has the form 

C> max I(X 1 ;Y 2 ,Y 3 \X 2 ), (1) 

p{x 1 )p(x 2 )p(y 2 \y2,X2) 

where the maximum is over all pmfs p(xi)p(x 2 )p(y 2 \y 2 , x 2 ) such that I(X 2 ; Y 3 ) > I(Y 2 ; Y 2 \X 2: Y 3 ). This 
lower bound was established using a block Markov coding scheme — in each block the sender transmits 
a new message, and the relay compresses its received signal and sends the bin index of the compression 
index to the receiver using Wyner-Ziv coding [13]. Decoding is performed sequentially. At the end of 
each block, the receiver first decodes the compression index and then uses it to decode the message sent 
in the previous block. Kramer, Gastpar, and Gupta [14] used an extension of this scheme to establish 
a compress-forward lower bound on the capacity of general relay networks. Around the same time, 
El Gamal, Mohseni, and Zahedi flT5l put forth the equivalent characterization of the compress-forward 
lower bound 

C> max mm{I{X 1 ;Y 2 ,Y 3 \X 2 ),I(X 1 ,X 2 ;Y 3 )-I{Y 2] Y 2 \X 1 ,X 2 ,Y 3 )}. (2) 

P(X 1 )p(x2)p(y2\y2,X2) 

As we will see, this characterization motivates a more general way to extend compress-forward to 
networks. 
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In this paper, we describe a noisy network coding scheme that extends and unifies the above results. On 
the one hand, the scheme naturally extends compress-forward coding to noisy networks. The resulting 
inner bound on the capacity region extends the equivalent characterization in ©, rather than the original 
characterization in (Q]). On the other hand, our scheme includes network coding and its variants as special 
cases. Hence, while the coding schemes for deterministic networks and erasure networks can be viewed as 
bottom-up generalizations of network coding to more complicated networks, our coding scheme represents 
a top-down approach for general noisy networks. 

The noisy coding scheme employs block Markov message repetition coding, relay signal compression, 
and simultaneous decoding. Instead of sending different messages over multiple blocks and decoding one 
message at a time as in previous compress-forward coding schemes ifTUl . |[T4l . the source transmits the 
same message over multiple blocks using independently generated codebooks. Although a similar message 
repetition scheme is implicitly used in the time expansion technique for cyclic noiseless networks (51 
and nonlayered deterministic networks JH, our achievability proof does not require a two-step approach 
that depends on the network topology. The relay operation is also simpler than previous compress- 
forward schemes — the compression index of the received signal in each block is sent without Wyner-Ziv 
binning. After receiving the signals from all the blocks, each destination node performs simultaneous 
joint typicality decoding of the messages without explicitly decoding the compression indices. As we 
will demonstrate, this results in better performance than previous schemes in lfl4l . lfT6l . ifTTl . lfT8l . |[T9l 
for networks with more than one relay node or multiple messages. 

The simplicity of our scheme makes it straightforward to combine with decoding techniques for 
interference channels. Indeed, the noisy network coding scheme can be viewed as transforming a multi- 
hop relay network into a single-hop interference network where the channel outputs are compressed 
versions of the received signals. We develop two coding schemes for general multiple source networks 
based on this observation. At one extreme, noisy network coding is combined with decoding all messages, 
while at the other, interference is treated as noise. 

We apply these noisy network coding schemes to Gaussian networks. For the multiple-source multi- 
cast case, we establish an inner bound that improves upon previous capacity approximation results by 
Avestimehr, Diggavi, and Tse (H and Perron j20ll with a tighter gap to the cutset bound. We then show 
that noisy network coding can outperform other specialized schemes for two-way relay channels |[T6ll , 
IfTTl and interference relay channels |[T8l . |fl9l . 

The rest of the paper is organized as follows. In the next section, we formally define the problem of 
communicating multiple sources over a general network and discuss the main results. We also show that 
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previous results on network coding are special cases of our main theorems and compare noisy network 
coding to other schemes. In Section |InJ we present the noisy network coding scheme for multiple-source 
multicast networks. In Section |IVJ the scheme is extended to general multiple-source networks. Results 
on Gaussian networks are discussed in Section [V] 

Throughout the paper, we follow the notation in ll2TTl . In particular, a sequence of random variables 
with node index k and time index % G [1 : n] is denoted as X% = (X k i, ■ ■ ■ ,X kn ). A set of random 
variables is denoted as X{A) = {X k : k G ^4}. 

II. Problem Setup and Main Results 

The iV-node discrete memoryless network (DMN) dlfcLi X k ,p(y N \x N ), Y\ k= i 34) depicted in Fig- 
ure Q] consists of N sender-receiver alphabet pairs (X k , y k ), k G [1 : N] := {1, . . . , N}, and a collection 
of conditional pmfs p(yi, . . . , Un\x\, ■ ■ • , xn). Each node k G [1 : N] wishes to send a message M k to 
a set of destination nodes, V k C [1 : N\. Formally, a (2 nRl , . . . ,2 nRN ,n) code for a DMN consists of 
iV message sets [1 : 2 nRl ], . . . , [1 : 2 nRN ], a set of encoders with encoder k G [1 : N] that assigns an 
input symbol x^i to each pair (m^, y 1 ^ 1 ) for i € [1 : n], and a set of decoders with decoder d € U^ =1 T>k 
that assigns message estimates {rh^d '■ k € Sd) to each (y^,mrf), where Sd ■= {k : d € V^} is the set of 
nodes that send messages to destination d. For simplicity we assume d £ Sd for all destination nodes. 

We assume that the messages M^, k G [1 : N], are independent of each other and each message is 
uniformly distributed over its message set. The average probability of error is defined as 

P e (n) = P{M kd + M k for some d G V k , k G [1 : N}}. 

A rate tuple . . . , Rjy) is said to be achievable if there exists a sequence of (2 nRl , . . . , 2 nR,N , n) 

(n) 

codes with P e —> as n — > oo. The capacity region of the DMN is the closure of the set of achievable 
rate tuples. 

We are ready to state our main results. 

Multiple-source multicast networks: In Section [Till we establish the following noisy network coding 
theorem for multicasting multiple sources over a DMN. The coding scheme and techniques used to prove 
this theorem, which we highlighted earlier, constitute the key contributions of our paper. 

Theorem 1: Let V = V x = • • • = V N . A rate tuple (R x , . . . , R N ) is achievable for the DMN p(y N \x N ) 
if there exists some joint pmf p(q) Y[ k =i P{x k \q)p(y k \y k , x k , q) such that 

R(S) < min I(X(S); Y(S C ), Y d \X(S c ), Q) - I(Y(S); Y(S)\X N ,Y(S c ),Y d , Q) (3) 
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for all cutsets S C [1 : N] with S c n £> / 0, where R(S) = Y, k eS R k- 

This inner bound has a similar structure to the cutset outer bound given by 

R(S) <I(X(S);Y(S C )\X(S C )) (4) 

for all S C [1 : N] with S c n V ^ 0. The first term of dU, however, has F replaced by the "compressed" 
version Y. Another difference between the bounds is the negative term appearing in ©, which quantifies 
the rate requirement to convey the compressed version. In addition, the maximum in (0) is only over 
independent X N . 

Theorem Q] can be specialized to several important network models as follows: 
Noiseless networks: Consider a noiseless network modeled by a weighted directed graph Q = (Af,£,C), 
where J\f = [1 : N] is the set of nodes, £ C [1 : N] X [1 : N] is the set of edges, and C = {C jk G R + : 
(j, k) G £ } is the set of link capacities. Each edge (j, k) G £ carries an input symbol Xjk G Xjk with 
link capacity Cjf. = log|A^fc|, resulting in the channel output at node k as Yjj. = {X,^ : (j, fc) G <S}. 
By setting Y k = Ife for all k and evaluating Theorem Q] with the uniform pmf on X N , it can be easily 
shown that inner bound ® coincides with the cutset bound, and thus the capacity region is the set of 
rate tuples (R\, . . . , Rn) such that 

R(S)< Y, ^ 

This recovers previous results in for the single-source case and for the multiple-source case. 

Relay channels: Consider the relay channel p(y2,V3 \x±, X2). It can be easily shown that the inner bound © 
reduces to the alternative characterization of the compress-forward lower bound in ©. 

Erasure networks: Consider the erasure multiple-source multicast network in which the channel output 
at node k 6 [1 : N] is Yj. = {Yjk : j € [1 : N]}, where Yj^ = e if it is erased, and = Xj, otherwise. 
Assume further that the network erasure pattern is known at the destination nodes. Taking Y^ = Yk, 
k € [1 : N] and the uniform pmf on X N as in the noiseless case, inner bound © reduces to 

R(S) < J^(log 1^1(1 - P{link (j, k) is erased for all k G «S C })) . (6) 

It can be also shown that the inner bound coincides with the cutset bound and thus characterizes the 
capacity region. This recovers the previous result in 10. 

Deterministic networks: Suppose Y^ = gk(Xi, . . . ,Xjy), k G [1 : N]. By setting Y^ = k G [1 : N], 
Theorem [T] implies that a rate tuple {R\, . . . ,Rn) is achievable for the deterministic network if there 
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exists some pmf p(q) Yl k=1 p(xk\q) such that 



R(S) < I(X(S);Y(S C )\X(S C ), Q) = H(Y(S C )\X(S C ), Q) 



(7) 



for all <S C [1 : N] with S c n V ^ 0. This recovers previous results in (H for the single-source case 
and in |[20l for the multiple-source case. Note that the lower bound (O is tight when the cutset bound 
is attained by the product pmf, for example, as in the deterministic network without interference or 



Note that in all the above special cases, the channel output at node k can be expressed as a deterministic 
function of the input symbols (X±, . . . , Xn) and the destination output symbol Yd, i.e., 



Under this structure, the inner bound in Theorem Q] can be simplified by substituting Yf. = Yp. for 
k G [1 : N] in © to obtain the following generalization. 

Corollary 1: Let V = T>\ = ■ ■ ■ = T>n- A rate tuple (R\, . . . ,Rn) is achievable for the semideter- 
ministic DMN ([8]) if there exists some joint pmf p{q) Y\k=iP{ x k\o) such that 



for all SC[1:JV] with S c n V ^ 0. 

We also show in Appendix O that our noisy network coding scheme can strictly outperform the 
extension of the original compress-forward scheme for the relay channel to networks in lfl4l Th 3]. 

General multiple-source networks: We extend the noisy network coding theorem to general multiple- 
source networks. As a first step, we note that Theorem Q] continues to hold for general networks with 
multicast completion of destination nodes, that is, when every message is decoded by all destination 
nodes V = U-^jPfc. Thus, we can obtain an inner bound on the capacity region for the DMN in the 
same form as (O with V = U^ =1 T>k. 

This multicast-completion inner bound can be improved by noting that noisy network coding trans- 
forms a multi-hop relay network p(y N \x N ) into a single-hop interference network p(y N \x N ), where the 
effective channel output at decoder k is Yfc = Yk,Y\, . . . , Yn) and the compressed channel outputs 
(Yi, . . . , Yjv) are conveyed to decoders with some rate penalty. This observation leads to a modified 
decoding rule that does not require each destination to decode unintended messages correctly, resulting 
in the following improved inner bound. 

March 15, 2010 DRAFT 




Y k = 9dk(X 1: X N , Y d ) for every k G [1 : N] and d G V. 



(8) 



R{S) <I(X(S);Y(S C )\X(S C ),Q) 



(9) 



s 



Theorem 2: A rate tuple {R\, ... ,Rn) is achievable for the DMN if there exists some joint pmf 
p(<l)Ilk=iP(xk\q)p(yk\yk,Xk,q) such that 

R(S)< min I(X(S);Y(S c ),Y d \X(S c ),Q)-I(Y(S);Y(S)\X N ,Y(S c ),Y d ,Q) (10) 

for all cutsets S C [1 : N] with S c n £>(<S) / 0, where V(S) := U keS V k . 
The proof of this theorem is given in Subsection IIV-AI 

As an alternative, each destination node can simply treat interference as noise rather than decoding it. 
Using this approach, we establish the following inner bound on the capacity region. 

Theorem 3: A rate tuple . . . ,Rn) is achievable for the DMN if there exists some joint pmf 
Hk=i P( u k, x k \q)p(y k \y k ,u k , q) with 



R(T) <I(X(T), U(S); Y(S c ),Y d \X(T c ), U(S C ), Q) - I(Y(S); Y(S)\X(S d ), U N , Y(S c ),Y d , Q) (11) 
for all SQ[l:N],d€ V(S), and «S n S d C T C S d such that S c n V{S) + 0, where T c = S d \T. 



Unlike the coding schemes in Theorems Q] and [2] where each node maps both its own message and the 
compression index to a single codeword, here each node applies superposition coding ll22l for forwarding 
the compression index along with its own message. (Note that when a node does not have its own message 
and it acts only as a relay, there is no difference in the relay operation from the previous schemes.) The 
details are given in Subsection IIV-BI 

Gaussian networks: In Section El we present an extension of the above results to Gaussian networks and 
compare the performance of noisy network coding to other specialized coding schemes for two popular 
Gaussian networks. 

Consider the Gaussian network 



where G € W NxN is the channel gain matrix and Z N is a vector of independent Gaussian random 
variables with zero mean and unit variance. We further assume average power constraint P on each 
sender X k . 

In Subsection |V-A[ we establish the following result on the multicast capacity region of this general 
Gaussian network. 

Theorem 4: Let V = T>\ = ■ ■ ■ = T>jq. For any rate tuple {R\, . . . , Rjy) in the cutset outer bound, 
there exists (R±, . . . , R' N ) in the inner bound in Theorem Q] for the AWGN network (fT2l ) such that 



Y N = GX 1S + Z 



■N 



(12) 
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for all S C [1 : AT] with S c n V ^ 0. 

This theorem implies that the gap between the cutset bound and our inner bound is less than or equal to 
(iV/4) log(2iV) for N > 3, regardless of the values of the channel gain matrix G and power constraint P. 

We also demonstrate through the following two examples that noisy network coding can outperform 
previous coding schemes, some of which are developed specifically for these channel models: 
Two-way relay channel ( Subsection \V-B\) : Consider the AWGN two-way relay channel 

Y\ = g 2 \X 2 + g 3 \X 3 + Z 1 , 

Y 2 = g i2 X 1 +g 32 X 3 + Z 2 , (13) 

Y 3 = gi 3 Xx + #23^2 + Z 3 , 

where the channel gains are g± 2 = g 2 \ = 1, g\ 3 = g 3 \ = dr^l 2 and g 23 = g 32 = (1 — d)~ 7 / 2 , and 
d € [0, 1] is the location of the relay node between nodes 1 and 2 (which are unit distance apart). Source 
nodes 1 and 2 wish to exchange messages reliably with the help of relay node 3. Various coding schemes 
for this channel have been investigated in ifToll , ifTTIl . In Figure [2j we compare the performance of noisy 
network coding (Theorem 2) to amplify-forward (AF) and an extension of compress-forward (CF) for 
d € [0, 1/2] and 7 = 3. Note that noisy network coding outperforms the other two schemes coinciding 
with the compress-forward only when the relay is midway between nodes 1 and 2 (d = 1/2) and when 
it coincides with node 1 (d = 0). 

5.5 
5.25 
5 

B 475 

s 45 

^ 4.25 
4 

3.75 
3.5 

( 

Relay location d 

Fig. 2. Comparison of coding schemes for P = 10. 
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Interference relay channel ( Subsection W-Ci : Consider the AWGN interference relay channel with or- 
thogonal receiver components in Figure [3] 

Z4 Z3 




Z 5 

Fig. 3. AWGN interference relay channel. 
The channel outputs are 

Yj = gijXi + g 2j X 2 + Zj, j = 3, 4, 5, 

where gij is the channel gain of link Source node 1 wishes to send a message to destination 

node 4, while source node 2 wishes to send a message to destination node 5. Relay node 3 helps 
the communication of this interference channel by sending some information about I3 over a common 
noiseless link of rate Ro to both destination nodes. In Figure 01 we compare noisy network coding 
(Theorems 2 and 3) to compress-forward (CF) and hash-forward (HF) in |[T9ll . The curve representing 
noisy network coding depicts the maximum of achievable sum rates in Theorems [2] and [3] Note that, 
although not shown in the figure, Theorem 3 alone outperforms the other two schemes for all channel gains 
and power constraints. At high signal-to-noise ratio (SNR), Theorem 2 provides further improvement, 
since decoding other messages is a better strategy when interference is strong. 

III. Noisy Network Coding for Multicast 

To illustrate the main idea of the noisy network coding scheme and highlight the differences from 
the standard compress-forward coding scheme iflOl . lfl4l . we first prove Theorem Q] for the 3-node relay 
channel and then extend the proof to general multicast networks. 

Let x fci denote (x fc (j _ 1)n+1 , . . . ,x k ,j n )> 3 G [1 : b\; thus x b k n = (x kl , . . . ,x kinb ) = (x fc i, . . . ,x fcb ) = 
x£. To send a message m € [1 : 2 ], the source node transmits xij(m) for each block j G [1 : 6]. 
In block j, the relay finds a "compressed" version y 2 j of the relay output y 2 j conditioned on X2j, and 
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, i i i i i i i i 

5 10 15 20 25 30 35 40 

Power constraint P (in dB) 



Fig. 4. Comparison of coding schemes for #14 = 325 = 1, gib = P24 = gis = 0.5, gia = 0.1. 

transmits a codeword x.2,j+i$2j) in the next block. After b block transmissions, the decoder finds the 
correct message m G [1 : 2 nbR ] using (y"3i , . . . , y3&) by joint typical decoding for each of b blocks 
simultaneously. The details are as follows. 

Codebook generation: Fix p(xi)p(x2)p(y2\y2, ^2)- We randomly and independently generate a codebook 
for each block. 

For each j G [1 : b], randomly and independently generate 2 nbR sequences xij(m), m G [1 : 2 nbR ], 
each according to YYi=i Vx x (x 1 ^j_ 1 - jn+i ). Similarly, randomly and independently generate 2 nR ' 2 sequences 
x 2 j(/ i _i), Ij-i G [1 : 2 nR % each according to n?=iPX 2 (^2,(j-l)n+i)- For each X2j(lj-l), G [1 : 
2 ni?2 ], randomly and conditionally independently generate 2 nR2 sequences y2j(lj\lj-i), lj G [1 : 2 nR2 ], 

each according to Yl2=i Py 2 \x 2 i.y2,(j-l)n+i\ x 2,(j-i)n+i 
This defines the codebook 

Cj = {^(m),^^-!),^^-!) : m G [1 : 2 nfefl ],/ J ,/ i _ 1 G [1 : 2 n « 2 ]} 

for j G [1 : 6]. 

Encoding and decoding are explained with the help of Table U 
Encoding: Let m be the message to be sent. The relay, upon receiving y 2j - at the end of block j G [1 : b], 
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Block 


1 


2 


3 


6-1 


6 


x 1 


xu(m) 


xi 2 (to) 


xi 3 (m) 


xi i6 _i(m) 


xib(m) 




y 2 i(ii|i),ii 




£23(^2)^3 ■ 


y2,6-l(^6-l \h-2),h-l 


y2b{h \lb-l), lb 


x 2 


X2l(l) 


X 2 2^l) 


X23(?2) 


X2,6-l(Z6-2) 




Y 3 














m 



TABLE I 

Noisy network coding for the relay channel. 



finds an index lj such that 

{y2j(lj\lj-i),y2j,*2j(lj-i)) e 

where Iq = 1 by convention. If there is more than one such index, choose one of them at random. 
If there is no such index, choose an arbitrary index at random from [1 : 2 nR2 ]. The codeword pair 
(jc.ij(m),X2j(lj-i)) is transmitted in block j G [1 : b\. 

Decoding: Let e > e'. At the end of block b, the decoder finds a unique message rh € [1 : 2 rafcfl ] such 
that 

(xi i (m),y2 i (/ i |f j -i),x 2j (/ i _i),y 3j ) G T e (n) for all jG[l: 6] 

for some li, Z 2 , ■ ■ . , Z&. If there is none or more than one such message, it declares an error. 

Analysis of the probability of error: Let M denote the message sent at the source node and Lj denote 
the indices chosen by the relay at block j 6 [1 : b\. Define 

b 

£0 := U{(Y 2i (Z i |L i _ 1 ),X 2i (L i _ 1 ),Y 2i ) 7^ for all /,}, 

£m '■= { (Xlj (m),Y 2i (/ i |/ j _i),X 2j (/ j _i),Y 3j ) G 7; (n) , j £ [1 : 6] for some Zi, Z 2 , . . . , l b }. 

To bound the probability of error, assume without loss of generality that M = 1. Then the probability 
of error is upper bounded by 

P(£) < P(£ ) + P(£ C n E{) + P(u m ^£ m ). 

By the covering lemma (HI, P(£o) — >■ as n — >• 00, if i? 2 > J(Y 2 ;Y2|X 2 ) + (5(e')- By the conditional 
typicality lemma (211, P{£q n f f ) -> as n 00. 
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To bound P(U m ^i<? m ), assume without loss of generality that (L\, . . . , = (1, . . . , 1); recall the 
symmetry of the codebook construction. For j G [1 : b], m G [1 : 2 nbR ], and € [1 : 2 rti?2 ], define 

the events 

Ajfalj-ulj) := {(X li (m),Y 2j (/ i |Z i _ 1 ) ) X 2i (Z i _ 1 ) ) Y 3i ) G T £ (n) }. 

Then, 

P{£ m ) = P(U^ nj=i Aj(m,lj-i,lj)) 

b 

= X)II p ^( m ' Z i-i' Z i)) (14) 

/" j=l 

z b i=2 

where equality (fl4l) follows since the codebook is generated independently for each block j and the 
channel is memoryless. Note that if m / 1 and Zj_i = 1, then Xy(ra) ~ Yli=i Vx x i x i,(j-i)n+i) lS 
independent of (Y2j(lj\lj-i),~X.2j(lj-i)^3j) (given = Lj = 1). Hence, by the joint typicality 

lemma EL 

PiAjimJj^lj)) = P{(X ai (m),Y 2i (Z i |i i _i),X2 i (i i _i),Y 3i ) G 7^} 

< 2-n(/(Xi;y 2 ,y s |X a )-(5(e)) 

_. 2-"( / i- <5 (<:))_ (15) 

Similarly, if m / 1 and Zj_i / 1, then 

n 

(Xij(m),X 2 j(Zj_l), Y 2 j(Zj|/j_l)) ~ ]^PX 1 (^l,(j-l)n+j)Px 2 ,y 2 ( X 2,(i-l)n+j ; y2,(j-l)n+i) 

i=l 

is independent of Y 3 j (given Lj-i = = 1). Hence, by the joint typicality lemma 

If the binary sequence Z 6-1 has k Is, then by (fT5T ) and ([161 ), 

f[ P(^(m, < 2 -n(W 1+ (fr-l-*)I a -(6-l)*( 6 » 
i=2 
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Therefore 

b b 

e n p(ak ij-uh)) = e e n p (A-^> 

i* j=2 l b p-ij=2 

6-1 

' h - 1 

it fc=0 
6-1 



<- j2"(6-l-fc)fl22-"(fcA + (6-l-A;)/2-(6-l)5(e)) 

« b fe=o 

6-1 

= ^ 1 ^2- n ( fc/ i+( b - 1 - fc )( / 2-«2)-(6-l)5(e)) 



Zb k=0 
6-1 



Thus, 



< E Ef 6 fe *) 2~™( (6 ~ 1)( ^ n{Jl ' /2 ~* 2} ~ <5(e)) ) 

l b k=0 

_ 2^22^-1 . 2- n (( b - 1 )( min {- f i. / 2-R2}-5(e))) 



6 

m£l l b 3=1 



as n — > oo, provided that 



< -^WIl J 2 - £ 2 } - 5(e)) - ^R 2 . 



Finally, by eliminating i£ 2 > ^(^2! ^2^2) + S(e') and letting 6 — > 00, we have shown the achievability 
of any rate 

R < mm{I(X r ,Y 2 ,Y 3 \X 2 ), I{X 1 ,X 2 ;Y 3 ) - I(Y 2 ;Y 2 \X 1 ,X 2 ,Y 3 )} -5(e) -d(e'). 

This concludes the proof of Theorem 1 for the special case of the relay channel. 

We now describe the noisy network coding scheme for multiple-source multicast over a general DMN 
p(y N \x N ). For simplicity of notation, we consider the case Q = 0. Achievability for an arbitrary time- 
sharing random variable Q can be proved using the coded time sharing technique [21 J. 

Codebook generation: Fix Y\k=i P( x k)p(yk\yk, Xk)- We randomly and independently generate a codebook 
for each block. For each j £ [1 : b] and k € [1 : N], randomly and independently generate 2 nbRh x 2 nRk se- 
quences x fcij (m fc ,/ fcij _i), m k £ [1 : 2 nbRk ], l kd ^ £ [1 : 2 nRk ], each according to YYi=iPx k ( x k,U-i)n+i)- 
For each node k G [1 : N] and each x.kj( m kih m k S [1 : 2 nfcRfc ], j— 1 £ [1 : 2 nfifc ], randomly and 
conditionally independently generate 2 nRfc sequences ykj(hj\ m ki hj £ [1 : 2 ni?fc ], each according 

to n"=iPy t |x fc (yfc,(i-i)™+il x fc,a-i)n+i( m fc^fcj--i))- This defines the codebook 

Cj = {x kj (m h ,lkj-i),ykj(hj\rn k ,lkj-i) : m fc € [1 : 2 nW * fc ], Z fej -, e [1 : 2 nRfc ], A; € [1 : N}} 
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for j G [1 : b}. 

Encoding: Let (mi, . . . ,mjv) be the messages to be sent. Each node fc G [1 : iV], upon receiving at 
the end of block j G [1 : 6], finds an index such that 

{ykj{kj\rn k ,lk,j-i),ykj,Xkj{mk,k,j-i)) G 7^ n) , 

where Z^o = 1, G [1 : iV], by convention. If there is more than one such index, choose one of them at 
random. If there is no such index, choose an arbitrary index at random from [1 : 2 nRk ]. Then each node 
k G [1 : N] transmits the codeword x^- (m^ , h,j-i) in block j € [1 : b]. 

Decoding: Let e > e'. At the end of block b, decoder d G T> finds a unique index tuple (my, . . . ,rh,Nd), 
where mud G [1 : 2 nbRk ] forked and m^d = m d, such that there exist some (hj, . . . ,Inj), hj G [1 : 
2 nR «], k^d and Z dj = j G [1 : b], satisfying 

(xij(mid, iij-i), • • • ,XNj(mNd, Inj-i), 

yij(iij\rhid, h,j-i), ■■■ , yNj^N^rriNd, lN,j-i),ydj) G 7^ n ) 

for all j G [1:6]. 

Analysis of the probability of error: Let M^ denote the message sent at node k G [1 : N] and L^j, 
k G [1 : N], j G [1 : 6], denote the index chosen by node A; for block j. To bound the probability of error 
for decoder dgD, assume without loss of generality that (Mi, . . . , Mjy) = (1, . . . , 1) =: 1. Define 

b N 

£ o ■= U U{(^(^|l,^-i),X fej (l,L feJ _i),Y fej ) £T}r ] for all Z fej } 
j=ifc=i 

£m := {(Xij(mi,Zij_i), . . . ,X.Nj(mN,lN,j-i), 

Yi j (l lj \m 1 ,l 1 j- 1 ), . . . , Y Nj (l Nj \m N ,l Nj j-i), Y dj ) G T^ n) , 

j G [1 : 6], for some (h, . . . , 1 & ), where l dj = L dj ,j G [1 : b}}. 

Here, L, = (/ij, . . . , Inj) for j G [1 : 6]. Then the probability of error is upper bounded as 

P(£) < P(5 ) + P(£o n ^i c ) + P(U m#1 5 m ), (17) 

where m := (mi, . . . , m^r) such that m d = 1. As in the 3-node case, by the covering lemma, P(£o) — ^ 
as n — >■ oo, if Rk > I(Yk',Yk\Xk) + S(e'), k G [1 : iV], and by the conditional typicality lemma 
P(£ , Qn£'];) — >■ as n — >■ oo. For the third term, assume without loss of generality that Li = • • • = L& = 1, 
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where Lj := (Lij, . . . ,Lnj). Define the events 

•Aj(m,lj-i,lj) ■= {(Xij(mi,/ij_i), . . . ,X Nj (niN,lN,j-i), 

Yij{hj\mi,hj-i), . . . , Y N j(l N j\mN,lN,j~i), Y$) G 7^ (n) } 

for m ^ 1 and all lj. Then, 

P(£m) = P(Up nj =1 ^(m^^lj)) 

< 5^ P(n5 =1 A(m, 1,-1,1,)) 

p 

b 

= EII f Wm,i;-i^)) (is) 

i b i=i 

6 

<En p (A(m,ii-i,i,)), 

16 j=2 

where (fT8l) follows since the codebook is generated independently for each block j and the channel is 
memoryless. 

For each \ b and j G [2 : 6], let Sj(m, l fe ) C [1 : N] such that <S,-(m, l 6 ) = {/c : m k / 1 or / fcj _i ^ 1}. 
Note that «Sj(m, l b ) depends only on (m, L/-i) and hence we write it as <Sj(m, lj-i). We further define 
T(m) C [1 : N] such that T(m) = {k : 7^ 1}. From the definitions we can see that T(m) C 
Sj(m, lj-i) and d G <S|(m, l,_i) C T c (m). 

Define X,-(«S,(m, lj-i)) to be the set of X;y(rrtfc, lk,j-i), k G «5j(m, lj-i), where wifc and hj-i are the 
coiTesponding elements in m and l b , respectively. Similarly define Yj(6j(m, lj-i)) and Yj(<Sj(m, lj-i))- 
Then, by the joint typicality lemma and the fact that 

n 

(X(5 J -(m,lj_i)),Y(5,-(m,l i _i))) ~ n px *( Ifc »(?- 1 )»^)^*|^*^*.(i- 1 )*^l aJfc .(?"- 1 )»»+ i ) 

fce5 3 (m,l 3 _i) i=l 

is independent of (X(<SJ(m, lj-i)), Y(<S?(m, lj-i)), Y^) (given Lj_i = Lj = 1), we have 
P(„4j(m,lj_i,L;)) < 2- n ( /l ( ,s ( m ' 1 - 1 )) +/2 ( 5 ( m ' 1 ^ 1 ))- 5 ( £ », 

where 

h{S) :=I(X(S);Y(S c ),Y d \X(S c )), 

I 2 (S) :=J2l(Y k ;Y(S c U{k' G S : k' < k}), Y d , X N \X k ). 
kes 
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Furthermore, from the definitions of T(m) and Sj(m, if m / 1 with rrid = 1, then 

^ 2 -n(7 1 (S,-(m,I 3 ._ 1 ))+/ 2 (5,-(m,l 3 _ 1 ))-^)) 

< e 2~ n(/i( ^ (m ' ij - i))+/2(iSj(m ^- i)) ~' 5(e)) 

SC[1:/V] l J _i:5 J (m,l J _ 1 )=5 
T(m)CS,dGS c 

< y2 2- n(/i(5)+/2( ' s) - E ^ s ^- 5 ( e )) 

SC[1:JV] 
T(m)CS,deS c 

< 2 N-l 2 -n(mm s (h(S)+I 2 (S)-j: kes R k -8(e))) ^ 

where the minimum is over S C [1 : N] such that 7~(m) C 5 and d G S c . Hence, 

b 

E EII p (AV, 

m^l p ,=2 

= EEEII p (A(m, 

= E En E p (A(m, i,-i,i,)) 

m^l 1„ J=2L,_ a 

- e En ( ^2" n ( /i ^( m ' ij - i )) +/2 ( 5 ^ m ' ij - 1 )^' 5 ^) 

m^l l b ,=2 

< ^ 2^r^R k2 T, k ^ d nR k2 (N-i)(b-i) 2 n(-(b-i)mm s (h(S)+l 2 (S)-^ (19) 

TC[1:JV] 

where the minimum in (fT9l is over 5 C [1 : N] such that T Q S,d € <S C . Hence, (TT9T ) tends to zero as 

n — > oo if 



\rc5,de<s c v kes / / 



E^ 



for all T C [1 : TV] such that T 7^ and d € T c . By eliminating > I(Y k ;Y k \X k ) + tf(e') and letting 
b — >• oo, the probability of error tends to zero as n — > oo if 

i?(T)< min U(5) + / 2 (5)-J]/(y fc ;y fc |X fc )j - (N - l)5(e>) - 5(e) 
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for all T C [1 : N] such that d G T c . Finally, note that 

h(S) -^I(Y k ;Y k \X k ) = -^I(Y k ;Y k \X N ,Y(S c ),Y d ,Y({k' e S : k' < k})) 
kes kes 

= -Y,HY k ;Y(S)\X N ,Y(S c ),Y d ,Y({k' £S:k'< k})) 
fees 

= -I(Y(S);Y(S)\X N ,Y(S c ),Y d ). 
Therefore, the probability of error tends to zero as n — > oo if 

R(T) < I(X(S); Y(S c ),Yd\X(S c )) - I(Y(S); Y(S) \X N ,Y(S c ),Y d ) - (N - l)d(e') - 5(e) (20) 

for all S,T C [1 : N] such that ^ T C S and d G S c . Since for every SC[1:JV] such that 5/0 
and d £ S c the inequalities with T C S are inactive due to the inequality with T = S in (1201 . the set 
of inequalities can be further simplified to 

R(S) < I(X(S);Y(S c ),Y d \X(S c ))-I(Y(S);Y(S)\X N ,Y(S c ),Y d ) - (N - l)S(e') - 6(e) (21) 

for all 5 C [1 : N] such that d G S c . Thus, the probability of decoding error tends to zero for each 
destination node d G V as n — > oo, provided that the rate tuple satisfies dJTJ. 

Hence, by the union of events bound, the probability of error for all destinations tends to zero as 
n — >• oo if the rate tuple {R\, ... , Rn) satisfies 

R(S) < d mm v I(X(Sy,Y(S c ),Y d \X(S c ))-I(Y(Sy,Y(S)\X N 

for all S C [1 : N] such that S c n V ^ for some n^iP( x fc)p(yfc|yfc) x fc)- Finally, by coded time 
sharing, the probability of error tends to zero as n — > oo if the rate tuple (R\, . . . ,Rn) satisfies 

R(S)< min I(X(S);Y(S c ),Y d \X(S c ),Q)-I(Y(S);Y(S)\X N ,Y(S c ),Y d ,Q) 
deS c nT> 

for all S C [1 : TV] such that S c n T> ^ for some Y\ k=l p(q)p(x k \q)p(y k \y k , x k , q). This completes the 
proof of Theorem Q] 

IV. Extensions to General Multiple-Source Networks 
A. Proof of Theorem 2 via Multicast Completion with Implicit Decoding 

We modify the decoding rule in the previous section to establish Theorem [2] as follows. 

Decoding: At the end of block b, decoder d G U^ =1 V k finds a unique index tuple {m kd : k G S d } such 
that there exist some (rh kd : k G S%) and . . . , Zjvj), where G [1 : 2 nbRk ] for k ^ d, rh dd = m d , 
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Ikj G [1 : 2 nRk ] for k 7^ d, and 1$ = 1$, j £ [1 : b], satisfying 

(x.ij(mid,h,j-l), ■ ■ ■ ,*Nj{m Nd , Inj-i), 

yij{hj\m ld , h,j-i), . . ■ ,yNj{lNj\m Nd ,, lN,j-i),y$) g 

for all j G [1 : b}. 

The analysis of the probability of error is similar to that for Theorem[TJin Section|III] For completeness, 
the details are given in Appendix |A] 

B. Proof of Theorem 3 via Treating Interference as Noise 

Codebook generation: Again we consider the case Q = 0. Fix Y\k=i P( u k, x k)p(yk\yk,Uk)- We randomly 
and independently generate a codebook for each block. For each j € [1 : b] and k € [1 : N], 
randomly and independently generate 2 nRk sequences u k j(l k j-i), G [1 : 2 nRk ], each according 

to Ui=iPU k (u k ,(j-i)n+i)- For each k e [1 : N] and each u kj (l k j-i), l k j-i G [1 : 2 nR »], randomly 
and conditionally independently generate 2 nbRk sequences x k j(m k \l k j-i), m k G [1 : 2 nbRk ], each 
according to ]J^ =1 p Xk \uS x k,(j-l)n+i\uk,(J-l)n+i(h,j-l))- For each k G [1 : N] and each u fci (I fc)j _i), 
Zfcj-i € [1 : 2 ni?fc ], randomly and conditionally independently generate 2 nRk sequences y k j(l k j\l k> j-i), 
l kj G [1 : 2 ni?fc ], each according to Y[^ =l PY k \ Uk {yK(i-i)n+i\ u k,{j-i)n+i{h,j-i))- This defines the code- 
book 

C j = {u kj (l kd - 1 ),x kj (m k \l k j-i),y kj (l kj \l k j-i) :m k e[l: 2 nbRk ],l kj ,l kJ ^ 1 e [1 : 2 nRk ],k e [1 : AT]} 
for j G [1 : 6]. 

Encoding: Let (mi, . . . , mjv) be the messages to be sent. Each node k G [1 : iV], upon receiving at 
the end of block j £ [1 : 6], finds an index l k j such that 

(ykj(hj\h,j-i),ykji u kj(lk,j-i)) g 7^ n \ 

where l k Q = 1, k E [1 : N], by convention. If there is more than one such index, choose one of them at 
random. If there is no such index, choose an arbitrary index at random from [1 : 2 nRk ]. Then each node 
k G [1 : N] transmits the codeword x. k j(m k \l kt j-i) in block j £ [1 : b]. 

Similarly as before, decoding is done by simultaneous joint typical decoding, however, since we are 
treating interference as noise, codewords corresponding to the unintended messages (m k : k € <S^) are 
discarded, which leads to the following. 
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Decoding: At the end of block b, decoder d G U^ =l T>k finds a unique index tuple (rhkd '■ k G Sd) such 
that there exist some (hj, . . . , iNj), where rhkd G [1 : 2 nbRk ] and k ^ d and m<^ = m^, hj G [1 : 2 n - Rfc ], 
A; 7^ d and = /^-, j G [1 : b], satisfying 

((x k j(rh k d\lk,j-i) ■ k GcSd),Uij(Zij_i),...,Ujvj(ijVj-l)) 

yijGij|Zij-i), • • • ,yArj(^jvj^jvj-i) ? y4j) g 7^ 

for all j G [1 : b}. 

The analysis of the probability of error is delegated to Appendix |B] 

V. Gaussian Networks 

We consider the additive white Gaussian noise (AWGN) network in which the channel output vector 
for an input vector X N is Y N = GX N + Z N , where G G K NxN is the channel gain matrix and Z N 
is a vector of independent additive white Gaussian noise with zero mean and unit variance. We assume 
average power constraint P on each sender, i.e., 

n 

^2 E { x h( m k,Y^~ x )) < nP 
i=i 

for all k G [1 : N] and m k G [1 : 2 nRk ]. For each cutset <S C [1 : N], define a channel gain submatrix 
G(S) such that 

Y(S C ) = G(S)X(S) + G'(S)X(S C ) + Z{S C ). 

In the following subsection, we prove Theorem [4] In Subsections IV-BI and IV-CI we provide the capacity 
inner bounds for the AWGN two-way relay channel and the AWGN interference relay channel used in 
Figures |2] and |U 

A. AWGN Multicast Capacity Gap (Proof of Theorem 4) 

The cutset outer bound for the AWGN multiple-source multicast network simplifies to the set of rate 
tuples such that 

+ imin{|5|,|5 c |}log(2|5|) (22) 

for all 5 C [1 : N] with S c n V ^ 0. To show this, first note that the cutset outer bound (@]) continues 
to hold with the set of input distributions satisfying E(X|) < P, k G [1 : N]. For each S C [1 : N] such 



I+^G(S)G(Sf 
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that S c n V ^ 0, we can further loosen the cutset outer bound as 

R(S)<I(X(S);Y(S C )\X(S C )) 

= h(Y{S c )\X{S c )) - h(Y(S c )\X N ) 

= h(G{S)X{S) + Z{S C )\X{S C )) - h{Y{S c )\X N ) 

= h(G{S)X{S) + Z{S C )) - h{Y{S c )\X N ) 



-log(2vre)l 5c l I + G(S)K x{s) G(S) rj 



■ log(27re) 



1 , 
< - log 
~ 2 E 



I + tr(K x(s) )G(S)G(Sf 



T 



I + \S\P - G(S)G(S) 
2\S\ ■ I + 2\S\- ■ G(S)G(S) T 



(23) 
(24) 



I+^G(S)G(Sf 



+ 



\S C 



log(2|5|), 



where K X (s) i s me covariance matrix of X(S), (l23l follows since tv(K)I — K is positive semidefinite 
for any covariance matrix K |[23l Theorem 7.7.3], and (l24l follows since tr(K X (s)) < l^!^ 3 ' from the 
power constraint. By rewriting (|24l as 



log 



I + \S\P ■ G(S)G(S) 



T 



log 



/+ \S\P ■ G{S) T G{S) 



and following similar steps, we also have 



R(S)<± log 

2 B 



I + ^G(S) T G(S) 
I + ^G(S)G(S) T 



+ i|ilog(2|5|) 
+ l|ilog(2|5|). 



On the other hand, the noisy network coding inner bound in Theorem Q] yields the inner bound 
characterized by the set of inequalities 



R(S)< -log 



I+-G(S)G(SY 



~2 



(25) 



for all S C [1 : N] with S c Pi V / 0. To show this, first note that by the standard procedure J2T|, 
Theorem Q] for the discrete memoryless network can be easily adapted for the AWGN network with 
power constraint, which gives the inner bound (0) on the capacity region with (product) input distributions 
satisfying E(Xf ) < P, k G [1 : N\. 

Let Q = and Xjt, fc G [1 : N], be i.i.d. Gaussian with zero mean and variance P. Let 

Y k = Y k + Z k , k€[l:N], 
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where 2%, k G [1 : N], are i.i.d. Gaussian with zero mean and unit variance. Then for each S C [1 : N] 
such that S c n P / and d G V, 



I(Y(S);Y(S)\X N ,Y(S c ),Y d ) < I(Y(S);Y(S)\X N ) 

= h(Y(S)\X N ) - h(Y(S)\Y{S),X N ) 

\s\ \s\ 

= — log(4vre) - — log(2vre) 
~ 2 ' 

where the first inequality is due to the Markovity (Y(5 C ), Y^) — > (X N ,Y(S)) — > Y(S). Furthermore, 



I(X(S);Y(S c ),Y d \X(S c )) > I(X(S);Y(S C )\X(S C )) 

= h(Y(S c )\X(S c )) - h(Y(S c )\X N ) 



^log(27re)l 5o l \2I + G(S)PG(S) T \ - ^ log(4vre) 



- log 

2 & 



I + ^G(S)G(S) T 



Therefore, by Theorem [T] a rate tuple (R\, . . . , R^) is achievable if 

T 



R(S)< ilog 



I+Zg(S)G(S) t 



for all 5 C [1 : JV] such that 5 c flP/0. 

Comparing the cutset outer bound (1221) and inner bound d25l) completes the proof of Theorem 4. 



B. AWGN Two-Way Relay Channels 

Recall the model for the AWGN two-way relay channel (fT3l) in Section ITT1 

Rankov and Wittenben [16] showed that the amplify-forward (AF) coding scheme results in the inner 
bound on the capacity region that consists of all rate pairs (R±,R2) such that 



1 ak + Ja 2 k -b 2 k 

i4<-log v - |, fee {1,2} 

for some a < ^ P / (gf 3 P + g 2 23 P + 1) , where m := 1 + ^l^f/^ ' a 2 := 1 + ^ff+f^ » 
b± ._ 2P »/3f 13912 ; and b2 ._ 2Pag 31 g2sg 21 _ Th alsQ showed that an extens i on Q f t h e original compress- 

forward (CF) coding scheme for the relay channel to the two-way relay channel results in the following 
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inner bound on the capacity region that consists of all rate pairs (R\,R 2 ) such that 

fg 2 3 P + (l + a 2 )g 2 2 P" 



Ri < C 



V 1 + <T 2 



for some 



a 2 > max 



1 + g 2 2 P)(l + g 2 3 P) - (g 12 g 13 P) 2 (1 + g 2 21 P)(l + 5 2 3 P) - ( ff2 i5 23 P) 2 



Specializing Theorem |2] to the two-way relay channel gives the inner bound that consists of all rate 
pairs (Ri,R 2 ) such that 

i? 1 <mm{/(X 1 ;F 2) F 3 |X 2 ,X3,Q) ! I(X 1 , X 3 ;Y 2 \X 2 ,Q) - I{Y 3 ;Y 3 \X X ,X 2 ,X 3 ,Y 2 ,Q)} 
R 2 <mm{I(X 2 ;Y 1 ,Y 3 \X l ,X 3 ,Q), I(X 2 ,X 3 ;Yx\Xx,Q) - I(Y 3 ;Y 3 \X 1 ,X 2 ,X 3 ,Y 1 ,Q)} 

for some p(q)p(xi\q)p(x 2 \q)p(x 3 \q)p(y 3 \y 3 , x 3 , q). By setting Q = and Y 3 = Y 3 +Z with Z ~ N(0,o- 2 ), 
this inner bound simplifies to the set of rate pairs (R X ,R 2 ) such that 

Ri < mm {C (4^±lI±j!M^ , C (g 2 2 P + g 2 32 P) - C(l/a 2 )} , 

R 2 < minjc ( g223P+ 1 (1 + + / )giljP ) > C ^i p i + ^V) " C(l/a 2 )} (26) 
for some a 2 > 0. 

C. AWGN Interference Relay Channels 

Recall the model for the AWGN interference relay channel with orthogonal receiver components in 
Figure 0] 

Djeumou, Belmaga, and Lasaulce [18], and Razaghi and Yu |fl9l showed that an extension of the 
original compress-forward (CF) coding scheme for the relay channel to the two-way relay channel 
results in the inner bound on the capacity region that consists of all rate pairs (R\, R 2 ) such that 

\g 2 l3 + (1 + a 2 )g 2 4 )P + (g 23 g u - g 2A gi 3 ) 2 P 2 ' 



Ri<C 
R 2 <C 



l + <J 2 + (g 2 23 + (l + a 2 )g 2 24 )P 

(g 23 + (1 + <? 2 )gl b )P + (ff!3ff25 - ffl5ff23) 2 ^ 2 

l + a 2 + (g 2 3 + (l + a 2 )g 2 5 )P 



for some 



2 > 1 f (ffl3ff24 ~ g 23 gi4,) 2 P 2 + Ql (ff!3ff25 ~ g23gl5) 2 ^ 2 + ^2 

° ~ - i ' maX I (9 2 uP + 9 2 2 ,p + 1) ' (g 2 15 P + g 2 25 P + 1) 
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where 

ai ■= {g 2 Vi + g 2 u)P + (g 23 + 9 2 2i )P + 1, 

a 2 ■= (3i3 + 9i 5 )P + (ffl 3 + &)P + 1- 
Razaghi and Yu jT9l generalized the hash-forward coding scheme ll24l . ifTTI for the relay channel to 

the interference relay channel, in which the relay sends the bin index (hash) of its noisy observation and 

destination nodes use list decoding. This generalized hash-forward scheme gives the inner bound on the 

capacity region that consists of the set of rate pairs (R\, R 2 ) such that 

'(gh + 9h)P + ^ 



Kx.Cl f 2 f ) + Rq-C^ + & )P + 1 



g 2 15 p + ij V (gf 5 P + iW 



for some a 2 > satisfying 



2 < 1 ■ ( (513524 ~ 5235u) P + gl (ffl3g25 ~ 92391$) P + «2 

a " - 1 ' mm I (5? 4 ^ + 5 2 2 4J p + 1) ' (gi 5 P + 4^ + 1) 

where a% and a 2 are the same as above. 

Specializing Theorem [2] by setting Y% = Y3 + Z with Z ~ N(0, ex 2 ) gives the inner bound that consists 
of all rate pairs (R\,R 2 ) such that 

R 1 < min jc(,LP) + i? - C(l/a 2 ), C ^£*±£±g&£ 
R 2 < min|c(, 2 2 5 P) + Ro - C(l/a 2 ), C ^ ^ + (1+^5)^ 
R 1 + R 2 < C((g 2 u + g 2 4 )P) + R - C(l/a 2 ), 

R 1 + R 2 <C + 92s)P + (1 + g2)(g ' 4 + g24)P + (gl3g24 ~ g23gl4 ^ p2 

V 1 + <7 2 

R 1 + R 2 < C((g 2 , + g 2 5 )P) + R - C(l/a 2 ), 

'(013 + 9 23 )P + (1 + ^ 2 )(5 2 2 5 + 5l 2 5 )^ + (523515 " 9l3g 25 ) 2 P 2 



R\ + i?2 < C , 

1 + <7 Z 

for some u 2 > 0. By the same choice of Y3, the inner bound in Theorem [3] can be specialized to the set 
of rate pairs (Ri,R 2 ) such that 

P < C ( 9 ^ P \ -4- /? C < ^3+g2 2 4)-P + A 
u ^nf ^13 + (! + (j2 )5i 2 4 )^ + (523514 - 5245i3) 2 i 32 ' 



l + <J 2 + (g 2 23 + (l + a 2 )g 2 24 )P 

Wi 5 p + i) V (g 2 i 5 P + i)° 2 J 
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(g^ 3 + (1 + <T 2 )gj 5 )P + (gl 3 ff25 ~ 9l592 3) 2 P Z 

i + ^ 2 + + (i + o 3 )^ 



VI. Concluding Remarks 

We presented a new noisy network coding scheme and used it to establish inner bounds on the capacity 
region of general multiple-source noisy networks. This scheme unifies and extends previous results on 
network coding and its extensions, and on compress-forward for the relay channel. We demonstrated 
that the noisy network coding scheme can outperform previous network compress-forward schemes. 
The reasons are: first, the relays do not use Wyner-Ziv coding (no binning index to decode), second, 
the destinations are not required to decode the compression indices correctly, and third, simultaneous 
decoding over all blocks is used. 

How good is noisy network coding as a general purpose scheme? As we have seen, noisy network 
coding is optimal in some special cases. It also performs generally well under high SNR conditions in 
the network. In addition, it is a robust and scalable scheme in the sense that the relay operations do 
not depend on the specific codebooks used by the sources and destinations or even the topology of the 
network. Noisy network coding, however, is not always the best strategy. For example, for a cascade 
of AWGN channels with low SNR, the optimal strategy is for the relay to decode the message and 
then forward it to the final destination. This simple multi-hop scheme can be improved by using the 
information from multiple paths and coherent cooperation as in the decode-forward scheme for the relay 
channel ifTOll and its extensions to networks |[25l . |Q"4"1 . Further improvement can be obtained by only 
partial decoding of messages at the relays |[T0l , and by combining noisy network coding with partial 
decode-forward to obtain the type of hybrid schemes in [10] and lfl4"l . 

Another important direction to improve noisy network coding for multiple sources is to use more 
sophisticated interference coding schemes, such as interference alignment |26l and Han-Kobayashi 
superposition coding ETl . 
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Appendix A 
Error Probability Analysis for Theorem [2] 

The analysis follows the same steps of the multicast case except that the union in the third error term 
of (fTTT ) is over all m such that (m^ : k G S d ) 7^ (1, ... , 1). Thus, 

P(U m £ m ) 

6 

<EEn p (A(m 5 W;)) 

m \b j=2 

< Y 2^mbRk 2 E M dnfl» 2 (JV-l)(b-l) 2 ™(~C'- 1 ) min s(/i(5)+/ 2 (5)-E fces ^-'5W)) ) ( 2 7) 
TC[1:7V] 

where the minimum in (|27T ) is over <S C [1 : ]V] such that T C 5 ,d G <S C . Hence, d27l ) tends to zero as 

n — > 00 if 



\Tf-.<! Jc.Qc V ^65 / 




for all T C [1 : N] such that T n 5 d ^ and d G T c . By eliminating > J(fj.; y fc |X fc ) + 5(e'), letting 
6 — > 00, and getting rid of inactive inequalities, the probability of error tends to zero as n — > 00 if 

R(S) < I(X(S);Y(S c ),Y d \X(S c )) - I(Y(S);Y(S)\X N ,Y(S c ),Y d ) - (N - l)6(e') - 5(e), (28) 

for all S C [1 : N] such that S n S d 7^ and cZ G 5 C . Thus, the probability of decoding error tends to 
zero for each destination node d G V as n — > 00, provided that the rate tuple satisfies (l28l ). Finally, by 
the union of events bound, the probability of error for all destinations tends to zero as n — > 00 if the 
rate tuple (R\, ... ,R^) satisfies 

R(S) < min I(X(S); Y(S C ), Y d \X(S c )) - I(Y(S);Y(S)\X N ,Y(S c ),Y d ) 

for all S C [1 : N] such that S c nV(S) ^ for some IlfcLi p( x k)p{i)k\yk, %k)- This completes the proof 
of Theorem [2] for Q = 0. The proof for the general Q follows by coded time sharing. 

Appendix B 
Error Probability Analysis for Theorem [3] 

Let Mfe denote the message sent at node k G [1 : N] and Lkj, k G [1 : N], j G [1 : b], denote the index 
chosen by node k for block j. To bound the probability of error for decoder d ET>, assume without loss 
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of generality that (Mi, . . . , M N ) = (1, . . . , 1) = 1. Define 

b N 

£ o ■■= U Ul^^l^-i),^^-!)^^) 07^ for all Z fej } 
j=ifc=i 

£m := { ({Xfcj(mfc|Zfcj_i) : A; G Sd},Uij(li,j-i), ■ ■ ■ , Ujvj(ijvj-i), 

Yy^l/ij-i), • • • , Y^^-i), Y^) G T e (n) , j G [1 : 6], 
for some (h, . . . , l b ), where Z dj = L dj ,j G [1 : 6]}. 

Here, m := (m* : k G <S d ) and lj = (Zy, . . . , Zjvj) for j G [1 : 6]. Then the probability of error is upper 
bounded as P(£) < P(£q) + P(£q H £J) + P(U m ^i£ m ), where = 1 in m. By the covering lemma, 
P(£o) ->■ as n -> oo, if > J(Yfc;Yfc|C/fc) + <5(e'), A; G [1 : N], and by the conditional typicality 
lemma P(£q PI £f ) — >■ as n — >■ oo. For the third term, assume that Li = ■ ■ ■ = Lj, = 1. Define 

,Aj(m,lj_i,lj) := {({X fci (m fc |Z fc j_i) : k G Uy(Zij_i), . . . , UjVj(ZjVj-i), 
Y y (Z y |Zij_i), . . . , Yjv^ZjVjIZiVj-i), Y^) G 7; {n) } 
for m / 1 and all lj. Then, from similar steps to the multicast case, 

P(£ m )<^n P (A(m^-i^))- 

l b j=2 

For each l 6 and j G [2 : 6], let ^(l 6 ) C [1 : TV] such that ^-(l 6 ) = {fc : Z fcji _i + 1}. We further define 
T(m) C [1 : N] such that T(m) = {k : k G 5 d , m k ^ 1}. By definition, cZ G T c (m) n <S?(l,_i), where 
T c (m) := 5d\T(m). Then, by the joint typicality lemma, we can show that 

P(A,(m,L,_i,l,,)) < 2- n ( /l ( 5 ^- 1 )' r ( m ))+ /2 ( 5 ^- 1 )' r ( m ))- 5 ( e )), 

where 

h(S, T) := I(X((S U T) n 5 d ), C/(5); y(5 c ), y rf |X((5 c n T) n S d ), U(S C )), and 

/ 2 («S,T) := ^/(y. ; y(5 c U{fc' G S : k' < k}),Y d , X(S d ),U N \U k ). 
kes 

Furthermore from the definitions of T(m) and 5j(lj_i), if m / 1 with m d = 1, then 

^ 2 -n(7 1 (5,(l 3 _ 1 ),T(m))+/ 2 (5 3 (l J _ 1 ),r(m))-5(e)) 



< ^ 2 -n(I 1 (S,T(m))+I 2 (S,T(m))-j: keS R k -6(e)) 
SC[l:N]:deS" 

< 2 7V-l 2 -„(min s (/ 1 (5,r(m))+/ 2 (5,r(m))-E fces « fc -'5W)). 
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Hence, 

b 

EEIT p (A(m, 1,-1,1,)) 

m^l \ b j=2 

= E EE l[p(M^-uW) 

m^l l b It— i ,=2 

= E En E p (A(m, 1,-1,1,)) 

m^l 1„ ,=2 1 J _ 1 

- E EII ( E 2 ~ n(/l(s ' (l3 '" l),r(m))+l2(53 ' (l3 " l),T(m))_<s(e)) 

m^l 1„ j=2 

6 

< / 2 iV-l 2 -r l ( I nin 5 (J 1 (S,r(m))+/ 2 (5,r(m))-E fces -R*-fi(6)))'J 
m^l 1„ ,=2 

< 2 E ^r" 6Jt * 2 E ^^ n ^2 (Ar - 1)(6 ~ 1) 2 n (~( fe - 1 ) mins ( /l ( 5 ' r ) +/2 ( 5 ' r )-^^s^- 5 ( e ))), (29) 
Tcs d 

where the minimum in (|29l is over 5 C [1 : JV] such that d G S c . Hence, d29l tends to zero as n-> oo 
if , 

for all T C 5^ such that c/ G T c . By eliminating R k > I(Yk',Yk\Uk) + 5(e') and letting b — > oo, the 
probability of error tends to zero as n — > oo if 

R(T)< min (h{S,T) + h{S,T)- Vl(n ; n|f/ fe ) ) - (iV - l)5(e') - 5(e) 

for all T ^ S d such that d G T c . Finally, note that 

I 2 (5, T) - E ^ ^l^fc) = " E ^ ^( 5C )' Y d, Y (i k ' e 5 : fc' < fe})) 

fees kes 

= -^I(Y k ;Y(S)\X(S d ),U N ,Y(S c ),Y d ,Y({k' e S : k' < k})) 

= -I(Y(S); Y(S) \X(S d ), U N , Y(S c ),Y d ). 

Therefore, the probability of error tends to zero as n — > oo if 

R(T) < I(X((S U T) n S d ), U(S)- Y(S c ),Y d \X((S c n r c ) n S d ), £/(S c )) 

- I(Y(S);Y(S)\X(S d ),U N ,Y(S c ),Y d ) - (N - l)5(e') - 5(e) (30) 
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for all S C [1 : N] and T C S d such that d G <S C and d G T c . Since for every 5 C [1 : N], d € S c the 
inequalities corresponding to T Cj (5 D 5^) are inactive due to the inequality with T = S D 5^ in (l30l ). 
the set of inequalities can be further simplified to 

i?(T) < I(X(T),U(S);Y(S c ),Y d \X(T c ),U(S c )) 

- I(Y(S);Y(S)\X(S d ), U N ,Y(S c ),Y d ) - (N - l)S(e') - 5(e) (31) 

for all S C [1 : AT] and S n 5 d C T C S d such that d G S c , where T c = S d \T. Thus, the probability 
of decoding error tends to zero for each destination node d G V as n — > oo, provided that the rate tuple 
satisfies (|3TT) . By the union of events bound, the probability of error tends to zero as n — > oo if the rate 
tuple (Rx, ... ,Rn) satisfies 

R(T) <I(X(T), U(S);Y(S C ), Y d \X(T c ), U(S C )) - I(Y(S); Y(S)\X(S d ), U N , Y(S c ),Y d ) 

for all S C [1 : N], d G V(S), and S (1 S d Q T Q S d such that S c n 2?(«S) 7^ 0, where T c = S d \T 
for some Hfe=i P{ x k)p(i)k\yk, %k)- This completes the proof of Theorem [3] for Q = 0. The proof for the 
general Q follows by coded time sharing. 

Appendix C 

Comparison to a Previous Extension of the Original Compress-Forward Scheme 

For a DM single-source (node 1) multicast network with destination nodes T> C [2 : N], a hybrid 
scheme proposed by Kramer, Gastpar, and Gupta O Theorem 3] gives the capacity lower bound 

C > max min I(X X ; Y 2 N ,Y d \U^,X^), (32) 

cfcX> 

where the maximum is over p(x\) Y\k=2P( u k-> x k)p{i)k\u2 , Xk, Uk) such that 
I(Y(T); Y(T)\U^ X?,Y(T c ),Y d ) + ^ I(Y k ; X?\U?,X k ) 

k&T 

T 

< I(X(T);Y d \U(T),X(T c ),U d ,X d ) +^J((7(^);y r(t) |[/(^),I r(t) ) (33) 

t=i 

for all T C [2 : iV], all partitions {/C t }f =1 of [2 : N], and all r(t) £ [2 : N] such that r(i) fC t . The 
complements T c and /Cf are the complements of the respective 7" and /C t in [2 : N]. 

The hybrid coding scheme achieving lower bound d32l ) uses an extension of the original compress- 
forward scheme for the relay channel as well as decoding of the compression indices at the relays. The 
pure compress-forward scheme without decoding gives the capacity lower bound 

C>R*= maxmm I(Xx;Y 2 N ,Y d \X^ ), (34) 
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where the maximum is over all pmfs Y\ k= iP(xk)p(yk\xk) such that 

I(Y(T);Y(T)\X?,Y(T c ),Y d ) + J £ i I(X?;Y k \X k ) < I(X(T);Y d \X(T c ),X d ) 

k&T 

for all T C [2 : N] and T c = [2 : N]\T. This is identical to (32) with Uj = 0, j £ [2 : N]. 

In the following we show that the noisy network coding lower bound in Theorem [TJ is uniformly better 
than lower bound (l34l ) for every p(y2 \% ). By using similar steps to those in [15, Appendix C] and 
some algebra, lower bound (|34l ) can be upper bounded as 

7?* < max min min I(X r , Y 2 N , Y d \X?) + I(X(T); Y d \X(T c ), X d ) 

d£T>TC[2:N] 

- J(Y(T); Y{T)\X?, Y(T c ),Y d ) - £ I(Y k ; X?\X k ) 

k£T 

= maxmin min I(X U X(T); Y(T), Y d \X(T c ), X d ) - /(Y(T); Y(T)\X N , Y(T), Y d ) 

-I(X(T);Y(T c )\Y d ,X(T c ),X d ) -^I(Y k ;X 2 N \X k ), (35) 

fceT 

where the maximums are over p(x\) Y\ k=2 p{x k )p(y k \x k , Uk)- Here equality (I35T ) follows since 
/(X i; Y 2 N , Y d \X? ) + /(X(T); Y d \X(T c ), X d ) - 7(Y(T); Y(T)\X», Y(T c ),Y d ) 
= I(X 1 ;Y(T c ),Y d \X^) + I(X 1 ;Y(T)\X 2 N ,Y(r c ),Y d ) 

+ I(X(T);Y d \X(T c ),X d ) - I(Y(T);Y(T)\X? ,Y(T c ),Y d ) 
= I(Xr,Y(T c ),Y d \X 2 N )+I(Xr,Y(T)\X 2 N ,Y(T c ),Y d )+I^ 

- I(X(T);Y(T C )\X(T C ), Y d , X d ) - 7(Y(T); Y \T)\X» ,Y \T C ) ,Y d ) 
= I(X 1 ,X(T) ] Y{T c ),Y d \X(T c ),X d ) + I(Xr,Y(T)\X^Y(T c ),Y d ) 

- I(X(T);Y(T C )\X(T C ), Y d , X d ) - 7(Y(T); Y \T)\X» ,Y (T) ,Y d ) 

= /(x 1 ,x(T) ; y(r c ),y rf |x(r c ),x (i ) + /(x 1 ,y(r);y(r)ix 2 iV ,y(r c ),y rf ) 

- I(Y(T); Y{T)\X U X%, Y(T c ),Y d ) - I(X(T); Y(T c )\X(T c ),Y d , X d ) 
-I{Y(T);Y{T)\X^Y{T c ),Y d ) 

= /(x 1 ,x(r);y(r c ),y d |x(r c ),x,) + /(x i; y(r)|x 2 Ar ,y(r c ),y(r),y rf ) 

- I(Y(T); Y{T)\X U X%, Y(T c ),Y d ) - I(X(T); Y(T c )\X(T c ),Y d , X d ) 

= /(x 1 ,x(T);y(r c ),y d |x(r c ),x,)-/(y(r) ; y(r)|x 1 ,xf,y(r c ),y d ) 

-I(X(T);Y(T c )\X(T c ),Y d ,X d ) 
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for all T C [2 : N], T c = [2 : N]\T and d G V, where the last equality follows from the Markovity 
{X U X{T C ), X d , Y(T C ), Y d ) -> (X(T),Y(T)) -> f (T). On the other hand, Theoremfflcan be simplified 
by setting Q = and R2 = ■ ■ ■ = Rn = as 

C> max min min II X(T); Y(T C ), Y d \X(T c ), X d ) (36) 

n»=iP(»*)P»»ll/».*») deC TC[2:7V] 

-/(y^iFfrii^.nr),^, 

where T c = [2 : iV]\7". Thus, Theorem [TJ achieves a higher rate than (l34l) with gap 

J(X(T); t(r c )|y d , X(T C ), X d ) + £ /(n; XflXfe) 

feeT 

for each d G £> and T C [2 : iV]. 

We now present a simple example for which noisy network coding performs strictly better than the 
general hybrid scheme (1321 . Consider a 4-node noiseless network, where V = {4}, R2 = R 3 = R4 = 0, 
and Y2 = Xi, Y 3 = X2, 14 = X3 are all binary. From (T5]), we know that that the noisy network coding 
lower bound achieves the capacity C = 1. On the other hand, applying (|32~1) to the above noiseless 
network we get 

I(X r ,Y 2 ,Y 3 ,Y 4 \U 2 ,U3,X 2 ,X 3 ) = I(X 1] Y 2 ,Y 3 \U 2 ,U3,X 2 ,X 3 ) (37) 

= I(X 1 ;Y 2 \U 2 ,U 3 ,X 2 ,X 3 ,Y 3 ) (38) 

where (f3Tb follows from the channel and (f38t follows from the Markovity X\ — >■ (C/2, ^3, -X3, I3) — > ^3- 
The constraint (|33l corresponding to T = {2} and r(l) = 4 is 

r 2 |^2, ^3, *2, X 3 , Y 3 , Y A ) + I(Y 2 ;X 3 \U 2 , U 3 , X 2 ) < I(X 2 ;Y 4 \U 2 , U 3 , X 3 ) + I(U 2 ; Yi\U 3 ), 

which can be simplified as 

I(X 1 ; Y 2 \U 2 , U 3 ,X 2 ,X 3 , Y 3 ) < I(U 2 ;X 3 \U 3 ) - I(Y 2 ; X 3 \U 2 , U 3 ,X 2 ) 

<I(U 2 ;X 3 \U 3 ) 

= 0, 

where the equality follows from U 2 — > U 3 — > -X3. Thus, the achievable rate of the hybrid scheme is zero 
for this particular example. It can be easily seen that our noisy network coding scheme outperforms the 
hybrid scheme for noiseless networks with more than two relays. Note that in general, due to decoding at 
the relay nodes, the hybrid scheme can sometimes perform better than the noisy network coding scheme 
without similar augmentation. 
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